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ABSTRACT 

Several studies have i-'ound that a realistic job 
preview reduces the rate of voluntary turnover. This study was 
conducted to determine whether the addition of a video-based test to 
a traditional selection process would improve selection decisions, 
increase the accuracy of job expectations, and reduce turnover. Bank 
tellers (N = 180) who had been selected using a traditional selection 
procedure (application, telephone screening interview, unstructured 
interview, and reference check) were compared with tellers (N-250) 
who had been selected using the same procedure plus a video-based 
selection test containing a realistic job preview. Applicants who did 
not pass any one of the selection instruments were not hired. 
Supervisors completed subjective ratings and collected objective 
performance measures using computer-generated reports for subjects at 
30-day intervals up to 180 days after hiring. Tellers hired after 
passing the video test exhibited significantly better performance 
than did tellers selected using the traditional process alone on the 
objective performance measures, but the two groups did not differ 
significantly on the subjective performance ratings. Tellers in the 
video group made fewer and less costly mistakes, took fewer sick 
days, and had less turnover than tellers in the traditional group. 
The results suggest that the job preview offered in the video test 
did not increase accuracy of expectations. (NB) 
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Improving Performance and Retention Through Video Assessment A Longitudinal Study 

Abstract 

Group A (n=180) tellers were selected using a traditional selection procedure, and Group B 
(n=250) tellers were selected using the same procedure plus a video-based selection test containing a 
realistic job preview. Group B exhibited significantly better performance on objective performance 
measures, but did not differ from Group A on subjective performance ratings. Turnover was significantly 
reduced in Group 3. The accuracy of the job preview did not differ significantly between the two groups. 
All performance measures correlated significantly with length of tenure. Turnover reduction was 
attributed to improved selection decisions rather than to more accurate job expectations. 

Introduction 

Several studies have summarized the research on the validity of alternative selection 
methods, and methods which simulate the job (e.g., assessment centers, work samples) have high 
validity coefficients (Hunter & Hunter, 1984; Schmitt, Gooding, Zoe & Kirsh, 1984; Reilly & Chao, 
1982; Muchinsky, 1986). Simulations replicate job situations and elicit applicant behaviors that may 
be exhibited if similar situations occur on the job, but are often quite costly to develop and administer. 
Video-based assessment is a relatively new selection technique which uses video to simulate job 
situations. Validity estimates obtained for video tests are comparable to those found for assessment 
centers (Mclntire & Thomas, 1990) and the tests are relatively inexpensive. 

The recruiting and selection process usually includes selling the applicant on the positive 
aspects of the job. This may give applicants inflated job expectations and may lead to increased 
turnover rates. Several studies have found that a realistic job preview reduces the rate of voluntary 
turnover (Ilgen & Seely, 1974; Meglino, DeNisi, Youngblood, & Williams, 1988; Reilly, Brown, Blood, 
& Malatesa, 1981). The video test selected for this study includes a realistic job preview. 

Hypothesis 1 : Tellers selected with the video test in addition to the traditional selection 
procedure (Group B) will perform better on the job (higher mean scores on subjective performance 
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measures and lower mean scores on objective performance measures) than will tellers selected using only 
the traditional selection procedure (Group A). 

Hypothesis 2 : Tellers who were administered the video test (Group B) will have more accurate 
job expectations than will those in Group A. 

Hypothesis 3 : Group B will have significantly less turnover than Group A. 

Method 

Subjects 

Data were collected for 430 tellers hired in 1989 by a bank in the Northeast. Group A consisted 
of 180 tellers hired using the traditional selection procedure. Group B consisted of 250 tellers hired 
using the traditional selection process plus the video test. Of the 430 tellers, 78 percent were female, 42 
percent were white, and 41 percent were black. The highest educational degree obtained by 81 percent 
of the total sample was a high school diploma. Eighty-three percent of the sample had no prior teller 
experience. 
Measures 

Teller Assessment Program (TAP) : TAP was developed in 1986 by Wilson Learning Corporation. 
Based on a traditional job analysis, TAP is an hour-long video simulation containing 37 multiple-choice 
questions. A five-minute introduction to the job of teller serves as part of the realistic job preview. TAP 
was originally validated using a predictive paradigm and found significant correlations (R=.47, j><.05, 
after correction for criterion unreliability) between video test scores and supervisory ratings 
(Assessment Designs, Intl., 1989). 

Preview Accuracy : The preview acccuracy questionnaire contained nine items. Respondents rate 
how strongly they agree or disagree with statements indicating that they were given an accurate 
description of various aspects of the job (e.g., customer interactions, money handling) during the hiring 
process. Items were rated on a five-point Likert scale, with 5 indicating "strongly agree," and 1 
indicating "strongly disagree." The score was derived by summing the ratings and dividing by the 
number of valid responses. 
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Subjective Performance Ratings : Using a five-point scale, supervisors were asked to rate how 
frequently the employee performed each of 48 beha^i^rs on a checklist. Scores were derived by 
summing the ratings and dividing by the number of nems rated. Supervisors also rated tellers' overall 
performance on a five-point scale as follows: Outstanding = 5, Above Expected = 4, Expected = 3, Below 
Expected = 2, Unacceptable=l. 

Objective Measures : These measures were each based on ? one-month period and included the 
number of times the teller was out of balance, total dollar amount over balance, total dollar amount 
under balance, and number of sick days. 
Procedure 

Group A (n=180) tellers were selected from January through April of 1989 using the traditional 
selection procedure, which involved an application, telephone screening interview, unstructured 
interview, and a reference check. Group A served as the closest approximation of a control group 
possible. Group B (n=250) tellers were selected from May through December of 1989 and were 
administered the same procedure. Group B was also administered the video test at a regional office 
prior to or following the employment interview. Applicants who did not pass any one of the selection 
instruments were not hired. Tellers completed the preview accuracy questionnaire between two and 
eight months after hire. 

Supervisors completed subjective ratings and collected objective performance measures using 
computer-generated reports for subjects at 30 day intervals up to 180 days after hire. Supervisors did not 
know teller's video test results. 

Results 

Chi-square analyses on gender, race, highest educational degree obtained, and previous 
experience revealed that the composition of Groups A and B did not differ significantly. 

Descriptive statistics of the video test scores were computed for Group B (M=80.10, 50=7.00, 
range = 70-99). Only those scoring at or above the cutoff (70) were hired. The number of tellers not 
hired based on their TAP score could not be determined because non-tellers were also tested. 
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A composite score was computed for each teller on each performance measure by averaging the 
individual's scores taken at each interval. The absolute values of the composite scores on "total dollar 
amount over balance" and "total dollar amount under balance" were summed to achieve a composite 
"total dollar error variance" score. Leaving the position for any reason (e.g., fired, resigned) was 
defined as turnover. 

Hypothesis 1 ; T-tests revealed no significant differences between the two groups on subjective 
performance measures. Mann-Whitney U-Wilcoxon Rank Sum W tests (used since objective measures 
scores were not normally distributed) revealed that Groups A and B did differ significantly on all 
objective performance measures: times out of balance (Z=-6.30, £<.0001), total dollar-error variance 
(Z=-5.39, e<.0001), and sick days (Z=^3.36, £<.001). See Table 1 for descriptive statistics. 

Hypothesis 2 : A t-test revealed that the groups did not differ significantly on mean preview 
accuracy scores nor on the variance. An additional t-test determined that the mean preview accuracy 
scores of tellers without experience in each of the groups did not differ significantly. 

Hypothesis 3 : Figure 1 shows the retention percentages for Groups A and B in the 14 months 
following hire. Chi-square tests comparing turnover percentages at each month after hire revealed 
that turnover rates for the groups were significantly different at 10 months (X 2 =3.86, df=l, £<.05), 11 
months (X 2 =4.15, df=l, £<.05), and 14 months (X 2 =4.03, df=l, £<.05). 

Since these results did not support the hypothesis regarding expectations but did show a 
turnover reduction, it was further hypothesized that more accurate selection decisions may have 
influenced turnover rates. The number of months on the job correlated positively with both supervisory 
performance ratings (r = .26 and .22, g < .01) and negatively with all objective measures of performance: 
out of balance, r = -.14, £ < .05; dollar error variance, r = -.22, p < .01; and sick days taken, r = .19, p < 
.01). Months on the job did not correlate significantly with job experience. 

Discussion 

This study was conducted to determine whether the addition of a video-based test to a 
traditional selection process would improve selection decisions, increase the accuracy of job 
expectations, and reduce turnover. Tellers hired after passing the video test exhibited significantly 
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bettor performance than did tellers selected using the traditional process alone on the objective 
performance measures, but the two groups did not differ significantly on the subjective performance 
ratings. Therefore, Group B tellers made fewer and less costly mistakes, and took fewer sick days 
which is a highly desirable and practically significant outcome. One possible explanation for the lack 
of significant differences between groups on the subjective measures is that rating errors and biases may 
have influenced supervisory ratings. 

The group selected using the video test did have less turnover; however, the results suggest that 
the job preview offered in the video test did not increase accuracy of expectations. One explanation of 
these results is that the instrument used to measure job expectations did not adequately assess 
expectations. It is also possible that applicants in Group A received an adequate and accurate job 
preview without viewing the video. 

The turnover rate did not differ for those with and those without prior .eller experience. Since 
those with previous teller experience should have accurate job expectations, this finding supports the 
conclusion that the accuracy of job expectations did not affect turnover. A study by Dean and Wanous 
(1984) supports the notion that realistic job previews alone do not produce turnover reductions. 

An alternative hypothesis is that the improved accuracy of the selection method for Group B 
impacted turnover rates. Evidence to si.pport this hypothesis is the significant correlation of months on 
the job with all subjective and objective performance measures. Given these results, the reduction of 
turnover seen in this study was more likely to have been the result of the selection of better performers 
than of more accurate job expectations. These results support previous research indicating that poor 
performers are more likely to leave the organization than are high performers (Dreher, 1982; Wells & 
Muchinsky, 1985; Kanfer, Crosby, & Brandt, 1988). 
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Table 1. Descriptive Statistics of Composite Scores on Performance Measures 





Mean 


SD 


Range 


Skew. 


Kurt. 


Total Supervisory Rating 
Group A 


3.84 
i on 


.54 

.0 1 


1.87-5.00 

0 A^ A OA 


-.76 


1.13 

fi7 


Overall Supervisory Rating 
Group A 
Group B 


3.29 


.77 

./O 


1.00-5.00 

l.UU - OXA) 


-.39 

AO 


.34 


Times Out of Balance 
Group A 
Group B 


4.83 
1.66* 


5.41 
2.03 


0-26 
0-15 


1.71 
2.88 


2.46 
12.13 


Total Dollar-Error Variance 
Group A 
Group B 


$ 164.80 
$ 54.89* 


275.64 
99.51 


0 - 2429 
0-1000 


5.46 
5.40 


41.66 
44.18 


Sick Days Taken 
Group A 
Group B 


1.98 
.86* 


3.14 
1.39 


0-18 
0-9 


2.80 
2.67 


8.94 
9.07 



* This Group B mean was Significantly lower than the Group A mean. 
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Figure 1. Retention Percentages of Groups A and B 



